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(57) Abstract 
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play pointing device (20) compris- 
ing a housing (21). an optical sen- 
sor mounted in the housing (21), and 
processing circuity receiving an output 
from the optical sensor for identifying 
a location on a display (16) pointed 
to by the pointing device (20) based 
on the optically sensible characteris- 
tics of at least one pattern at the lo- 
cation which is sensed by the optical 
sensor. A method of operating the dis- 
play pointing system (10) is also dis- 
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DISPLAY POINTING DEVICE 

FIELD OF THE INVENTION 

The present invention relates to display pointers which can be used to 
indicate location on a display, such as a computer display. 

BACKGROUND OF THE INVENTION 

Various types of display pointers are known in the art. These include 
devices such as "light pens" or toy firing devices which can be pointed at or brought into 
propinquity with a given location on a display, such as a CRT display, for sensing 
illumination features thereof Conventional light pens of this type require relatively 
complex synchronization circuitry for determining location on the display based on the 
time relationship of sensed illumination and of the time base of the display, including its 
synchronization pulses. 

Prior art pointers of the type described hereinabove often require 
feedback in order to achieve location sensing. In practice a sensor on the pointer feeds 
back a signal that it sensed on the screen to a timing circuit in a computer. The timing 
circuit determines the time relationship of the sensed signal to the vertical and horizontal 
video synchronization signals of the screen and thus determines the position of the 
pointer on the screen. 

The current state of the art is believed to be represented by the following 
U.S. Patents: 5,187,467; 5,184,115; 5,119,075; 4,608,601; 4,602,907; 4,190,851; 
3,944,988; 3,832,693; 3,758,717; 3,659,281; 3,594,608; 3,576,574; 3,551,896; 
3,543,240;3,506,875; 3,505,561; 5,572,251; 5,138,304; 5,134,389;5,051,736;4,956,639; 
4,922,236; 4,675,665; 4,591,841; 4,377,810 & 4,367,465. 

SUMMARY OF THE INVENTION 

The present invention seeks to provide an improved, relatively low cost 
display pointing device having multiple and varied applications. 

There is thus provided in accordance with a preferred embodiment of the 
present invention a display pointing device including: 

a housing; 

an optical sensor mounted in the housing; and 
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processing circuitry receiving an output from the optical sensor for 
identifying a location on a display pointed to by the pointing device based on the 
optically sensible characteristics of at least one pattern sensed by the optical sensor. 

There is also provided in accordance with a preferred embodiment of the 
present invention a display pointing method including: 

pointing an optical sensor mounted in the housing at a location on a 

display; 

receiving an output from the optical sensor; and 

identifying a location on the display pointed to by the pointing device 
based on the optically sensible characteristics of at least one pattern sensed by the optical 
sensor. 

In accordance with one embodiment of the present invention, the 
processing circuitry determines a location on the display pointed to by the pointing 
device based on the optically sensible characteristics of a plurality of patterns, which are 
seen sequentially by the optical sensor and which are optically distinguishable from each 
other. 

The at least one pattern may or may not be superimposed on a displayed 
image, as by eliminating lines in a displayed image. The at least one pattern may cover 
the entire display or only parts thereof 

The processing circuitry may employ pattern recognition circuitry which 
may be hard wired or embodied, wholly or partly in software. 

The pointer may be used with a CRT display or alternatively with other 
types of display such as liquid crystal displays. 

BRIEF DESCRIPTION OF THE DRAWINGS 

The present invention will be understood and appreciated more fully from 
the following detailed description, taken in conjunction with the drawings in which: 

Fig. 1 is a simplified illustration of a CRT pointing system constructed 
and operative in accordance with a preferred embodiment of the present invention; 

Figs. 2A, 2B and 2C illustrate the operation of a CRT pointing system of 

Fig. 1; 
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Figs. 3 A, 3B, 3C, 3D and 3E are illustrations of five coded patterns 
employed in the system of Figs. 2A - 2C and corresponding pulse trains sensed by a 
pointer viewing such patterns; 

Fig. 4 is a simplified sectional illustration of a preferred pointer 
constructed and operative in accordance with a preferred embodiment of the present 
invention; 

Figs. 5A and 5B together are an electrical schematic illustration of a 
preferred embodiment of the pointer of Fig. 4; 

Fig. 6 is a simplified illustration of a CRT pointing system constructed 
and operative in accordance with another preferred embodiment of the present invention; 

Figs. 7A, 7B, 7C, 7D and 7E illustrate the operation of a CRT pointing 
system of Fig. 6; 

Fig. 8 is a timing diagram related to the operation of the embodiment of 

Figs- 6 - 7E; 

Fig. 9 is a simplified illustration of a CRT pointing system constructed 
and operative in accordance with still another preferred embodiment of the present 
invention; 

Figs. lOA, lOB, IOC and lOD illustrate the operation of a CRT pointing 
system of Fig. 9; 

Figs. 11 A, IIB and IIC illustrate the operation of the embodiment of 

Figs. 9- lOD; 

Fig. 12 is a simplified illustration of a CRT pointing system constructed 
and operative in accordance with yet another preferred embodiment of the present 
invention; 

Fig. 13 is an illustration of the structure of the field of view of a trackable 
cursor employed in the system of Fig. 12; 

Figs. 14A and 14B are together a flowchart illustrating the operation of 
the system of Figs. 12 and 13; and 

Fig. 15 is an illustration of apparatus for remote pointing in accordance 
with an embodiment of the present invention. 
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LIST OF APPENDICES 

Appendix 1 is listing of software embodied in a microcontroller forming 
part of the circuitry of Fig. 5B; 

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS 

Reference is now made to Fig. 1, which is a simplified illustration of a 
CRT pointing system constructed and operative in accordance with a preferred 
embodiment of the present invention. The system comprises a conventional computer 10 
including a circuit board housing 12, a keyboard 14 and a display 16, such as a CRT ) 
display. In accordance with a preferred embodiment of the present invention, there is 
additionally provided a pointer 20 which can be used to point to a given location on the 
display 16, as figuratively illustrated in Fig. 2 A. ^ 

In accordance with a preferred embodiment of the present invention, the 
pointer 20 includes a hand held housing 21 within which is disposed an optical sensor 
which is preferably located at a tip 22 thereof The optical sensor is coupled by a wire 24 
or alternatively in a wireless manner to processing circuitry which may be located within 
the housing 21 or alternatively in a circuit board 26 located within circuit board housing 
12. 

Figs. 2A, 2B and 2C illustrate the operation of the CRT pointing system 
of Fig. 1 in accordance with one embodiment of the present invention in the context of a 
simple game in which the user bursts balloons by pointing at them. The user normally 
sees what is illustrated in Fig. 2A, however, in accordance with a preferred embodiment 
of the invention, intermittently and for a duration which preferably is so short that it 
cannot be seen by the user, each of the balloons is indicated by a differing pattern, as 
seen in Fig. 2B. This pattern is sensed by the optical sensor in the pointer 20 and is 
recognized by the processing circuitry downstream thereof Once a given pattern 
corresponding to a given balloon is recognized, the processing circuitry provides an 
output indication which causes a burst balloon image to appear on the display, as 
illustrated in Fig. 2C. 

Reference is now made to Figs. 3 A, 3B, 3C, 3D and 3E, which are 
illustrations of five coded patterns employed in the system of Figs. 2A - 2C and 
corresponding pulse trains sensed by a pointer viewing such patterns. Fig. 3 A illustrates a 
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blank screen which contains all scan lines, while Figs. 3B, 3C, 3D and 3E illustrate the 
patterns of balloons which are indicated by respective reference numerals 30, 32, 34 and 
36 in Fig. 2B. It is to be appreciated that preferably, the optical sensor senses the pulse 
train corresponding to each of the patterns without requiring synchronization with the 
vertical and horizontal synchronization pulses employed by the display. 

Reference is now made to Fig. 4, which is a simplified sectional 
illustration of a preferred pressure-actuated pointer 20 constructed and operative in 
accordance with a preferred embodiment of the present invention. The pointer preferably 
comprises a housing 40 having a spring biased retractable tip 42 in which is disposed an 
optical sensor 44, such as a Motorola MRD 3056 phototransistor. 

Retraction of tip 42 against the urging of a spring 46 is operative to close 
a switch, shown schematically at reference numeral 48. The output of optical sensor 44 is 
preferably supplied to processing circuitry located on a printed circuit board 50 which 
communicates via a cable 52 and a connector 54 with suitable circuitry within circuit 
board housing 12 (Fig. 1). 

Reference is now made to Figs. 5A and 5B, which together are an 
electrical schematic illustration of a preferred embodiment of the pointer of Fig. 4. It is 
seen that the optical sensor 44 is preferably a phototransistor such as an MRD 3056 
which outputs via an amplification circuit to a signal processing circuit Ul, such as a 
PIC16CR54RC-P microcontroller. The signal processing circuit is in turn coupled to 
connector 54, which is typically a DB15 connector which interfaces with a conventional 
game port of any suitable PC computer. 

Reference is now made to Fig. 6, which is a simplified illustration of a 
CRT pointing system constructed and operative in accordance with another preferred 
embodiment of the present invention. In this embodiment, rather than sensing a pattern, 
the optical sensor and associated processing circuitry senses a series of sequential 
patterns, the combination of which indicated a particular location on the display 1 16. 

Referring now to Figs. 7A, 7B, 7C, 7D and 7E and the timing diagram of 
Fig. 8, it can be seen that while the user sees the display shown in Fig. 7A, in accordance 
with a preferred embodiment of the invention, and as distinguished from the embodiment 
of Figs. 2A - 3F, sequentially and over a duration which preferably is so short that it 



5 

SUBSTITUTE SHEET (RULE 26) 



wo 98/35336 



PCT/IL97/00404 



cannot be seen by the user, each of the patterns of Figs. 7B, 7C, and 7D appears on the 
display at least on the portion of the display which the pointer is pointing to. 

The sequence of patterns sensed by the optical sensor in the polnier 20 
and is recognized by the processing circuitry downstream thereof Once a given pattern 
sequence corresponding to a given location is recognized, the processing circuitry 
provides an output indication which causes an illuminated symbol to appear on the 
display at the given location, as illustrated in Fig. 7E. 

Fig. 8 illustrates the identity of the pattern which is sensed by the optical ) 
sensor, aimed at the black pawn, at various times. Thus at time Tl, which corresponds to 
Fig. 7A, no pattern is sensed. At time T2, pattern C (Fig. 7B) is sensed. At time T3, 
pattern D (Fig. 7C) is sensed, at time T4, pattern B (Fig. 7D) is sensed and at time T5, ^ 
no pattern is sensed, as indicated in Fig. 7E. 

Reference is now made to Fig. 9, which is a simplified illustration of a 
CRT pointing system constructed and operative in accordance with still another 
preferred embodiment of the present invention. A standard computer application is 
shown on the display screen 1 16. 

Referring additionally to Figs. lOA, lOB, IOC and lOD it is seen that in 
contrast to the pointing systems described hereinabove, wherein each region has a unique 
pattern or patterns which is not sensitive to pen position within the region, here highly 
accurate vertical position sensing is provided. 

Fig. 10 A shows the screen 116 that is normally viewed by the user, while 
Figs. lOB and IOC illustrate sequential patterns 120, 122 which appear on the display 
screen 116 to provide position determination. The pattern 120 of Fig. 1 OB is a series of 
vertical lines which enable vertical position therealong to be sensed unambiguously, while 
the pattern 122 of Fig. IOC is a pattern which enables horizontal position to be sensed, 
albeit to a lesser degree of resolution than that realized for the vertical position. Fig. lOD 
illustrates the appearance of display screen 116 when the pointer is pointing at the "4" 
key. 

The pattern of Fig. IIA is a particularly useful arrangement of lines for 
unambiguously indicating vertical position, it being understood that various other 
arrangements may be employed. The pattern of Fig. 11 A is built of pairs of illuminated 



6 

SUBSTITUTE SHEET (RULE 26) 



wo 98/35336 



PCT/IL97/00404 



horizontal lines, each indicated by an asterix, separated by typically six lines which may 
or may not be illuminated in accordance with a given binary code. The six horizontal 
lines thus provide 64 different coded combinations. 

In order to minimize ambiguity, the pattern of Fig. IIA employs 
successive inversely coded arrangements. Thus the first group of six horizontal lines 
represents the binary number "5" and the successive group of six horizontal lines 
represents the binary number "58". The next group of six horizontal lines represents the 
binary number 6, and the next successive group of six horizontal lines represents the 
binary number "57". 

Reference is now made to Figs. 11 A, IIB and IIC, which illustrate the 
operation of the embodiment of Figs. 9 - lOD. Fig. 1 1 A shows a portion of the pattern of 
Fig. lOB and indicates, at reference numeral 80, a portion of the pattern which can be 
seen by a pointer 20 at a given pointer position. 

Fig. 1 IB indicates the code sequence sensed by the pointer 20 viewing the 
pattern portion indicated by reference numeral 80, while Fig. IIC indicates the code 
sequence sensed by the pointer 20 viewing a pattern portion indicated by a reference 
numeral 82. 

It may be appreciated from a consideration of Figs. 1 IB and 11 C that the 
coarse position of the pointer is determined by the identity of the coded 6-line pattern, 
while the precise position of the pointer is determined by counting the number of scan 
lines between the extreme illuminated scan lines that are sensed by the optical sensor. 
This indicates the extent of the field of view of the optical sensor. By dividing this 
number in half, the center of the field of view is determined and thus the precise position 
of the pointer is determined. The center of the field of view of the pointer is indicated as 
Yl and Y2 respectively in Figs. 1 IB and 1 IC. 

Once the precise vertical position of the pointer is determined, the 
horizontal position is determined by means of sensing the patterns shown in Fig. IOC, 
using the techniques described hereinabove with reference to Figs. 2B - 3E. 

Reference is now made to Fig. 12, which is a simplified illustration of a 
CRT pointing system constructed and operative in accordance with yet another preferred 
embodiment of the present invention. The pointing system of Fig. 12 is particularly 
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suitable for "drawing" on a display screen 100, using a pointer 102. The CRT pointing 
system of Fig. 12 preferably employs a multi-patterned trackable cursor dot 110, such as 

that shown in Fig. 13. 

Fig. 13 illustrates the structure of the trackable cursor 110 employed in 
the system of Fig. 12, showing preferably nine different sensing regions, each bearing a 
different pattern, which pattern can be distinguished one from the other by an optical 
sensor (not shown) in pointer 102 or by downstream circuitry associated therewith. 

Figs. 14A and 14B are together a flowchart illustrating the operation of 
the system of Figs. 12 and 13. When the pointer 102 is directed at one of the regions of 
the cursor dot 110 of Fig. 13, the pattern read by the optical sensor indicates the 
direction of intended movement of the cursor dot. Thus, as indicated in Figs. 14A and 
14B, if the sensed pattern is the C or central pattern, the X and Y positions of the cursor 
dot are not changed. 

If however, the optical sensor senses the pattern "N", the cursor is 
displaced upwardly in the next frame. Similar cursor displacements are indicated by 
sensing the patterns "S", "E", "W", "NW", "NE", "SW" and "SB", as indicated in Figs. 
14Aand 14B. 

Reference is now made to Fig. 15, which is an illustration of apparatus for 
remote pointing in accordance with an embodiment of the present invention. The pointer 
of Fig. 15 typically comprises a generally gun-shaped housing 140 having a lens 142 at 
the front thereof for viewing a given location on a display screen 143. An optical sensor 
144, such as a Motorola MRD3056 phototransistor, is provided at the focal plane of lens 
142. A trigger 146 is operative to close a switch, shown schematically at reference 
numeral 148. The output of optical sensor 144 is preferably supplied to processing 
circuitry located on a printed circuit board 150 which communicates via a cable 152 or 
alternatively in a wireless manner, and a connector 154 with suitable circuitry within 
circuit board housing 12 (Fig. 1). 

It will be appreciated by persons skilled in the art that the present 
invention is not limited by what has been particularly shown and described hereinabove 
but rather extends to equivalents of the structures and features described and shown as 
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known or obvious from the prior art. 
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CLAIMS 

1. A display pointing device comprising: 
a housing; 

an optical sensor mounted in the housing; and 

processing circuitry receiving an output from the optical sensor for 
identifying a location on a display pointed to by the pointing device based on the 
optically sensible characteristics of at least one pattern at said location which is sensed by 
the optical sensor. 

2. A device according to claim 1 and wherein said processing circuitry 
identifies the location on the display pointed to by the pointing device based on the 
optically sensible characteristics of a plurality of patterns, which are seen sequentially by 
the optical sensor and which are optically distinguishable from each other. 

3 A device according to either of the preceding claims and wherein said at 

least one pattern is superimposed on a displayed image. 

A device according to any of the preceding claims and wherein said at 
least one pattern is superimposed on a displayed image by eliminating lines in said 
displayed image. 

5- A device according to any of the preceding claims and wherein said at 
least one pattern covers generally an entire screen of said display. 

6- A device according to any of the preceding claims 1 - 4 and wherein said 
at least one pattern covers parts of a screen of said display. 

^- A device according to any of the preceding claims and wherein said 

processing circuitry employs pattern recognition circuitry. 
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g A device according to any of the preceding claims and also comprising a 

display. 

9. A device according to claim 8 and wherein said display is a CRT display. 



10. A device according to claim 8 and wherein said display is a liquid crystal 
display. 

) 

11. A display pointing method including: 

pointing an optical sensor mounted in the housing at a location on a 

display; ' 
receiving an output from the optical sensor; and 

identifying a location on the display pointed to by the pointing device 
based on the optically sensible characteristics of at least one pattern sensed by the optical 
sensor. 

12. A method according to claim 11 and wherein said step of identifying a 
location employs optically sensible characteristics of a plurality of patterns, which are 
seen sequentially by the optical sensor and which are optically distinguishable from each 
other, 

13. A method according to either of the preceding claims 11 and 12 and 
wherein said at least one pattern is superimposed on a displayed image. 

14. A method according to any of the preceding claims 11-13 and wherein 
said at least one pattern is superimposed on a displayed pattern by eliminating lines in 
said displayed image. 

15. A method according to any of the preceding claims 11-14 and wherein 
said at least one pattern covers generally an entire screen of said display. 
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16. A method according to any of the preceding claims 11-14 and wherein 
said at least one pattern covers parts of a screen of said display. 

17. A method according to any of the preceding claims 11-16 and wherein 
the determination step employs pattern recognition. 

18. A method system according to any of the preceding claims 11-17 and 
wherein said optical sensor views a CRT display. 

19. A method according to any of the preceding claims 11-17 and wherein 
said optical sensor views a liquid crystal display. 

20. A method according to claim 17 and wherein said pattern recognition 
employs an unambiguous pattern of scan lines. 
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